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Final  Report 

Collaborative  AFOSR  grant  FA9550-11-1-0151 

Project  Period:  from  July  01,  2012  to  December  31,  2013 

Project  Title:  Further  Development  in  the  Global  Resolution  of  Convex  Programs  with 
Complementarity  Constraints 

Principal  Investigators:  John  E.  Mitchell  (Rensselaer  Polytechnic  Institute)  and 
Jong-Shi  Pang  (University  of  Illinois  at  Urbana-Champaign  till  August  15,  2013;  substituted 
administratively  by  Angelia  Nedich  from  August  16,  2013  to  December  31,  2013) 

Project  Participants:  Yu-Ching  Lee  (University  of  Illinois  at  Urbana-Champaign); 
Bin  Yu  and  Lijie  Bai  (both  at  Rensselaer  Polytechnic  Institute).  These  students  have  suc¬ 
cessfully  defended  their  doctoral  dissertations  whose  topics  are  all  concerned  with  various 
aspects  of  the  project  supported  by  this  grant. 

Project  Summary:  This  collaborative  project  aims  at  the  study  of  the  global  res¬ 
olution  of  convex  programs  with  complementarity  constraints  (CPCCs),  which  form  a  large 
subclass  of  the  class  of  mathematical  programs  with  complementarity  constraints  (MPCCs). 
Despite  the  large  literature  on  the  local  properties  of  an  MPCC,  there  is  a  lack  of  systematic 
investigation  on  the  computation  of  a  globally  optimal  solution  of  these  constrained  opti¬ 
mization  problems,  or  in  the  case  where  such  a  solution  does  not  exist,  on  the  generation  of 
a  certihcate  demonstrating  the  solution  non-existence.  While  this  is  by  no  means  an  easy 
task,  the  pervasiveness  of  the  CPCC  in  applications  provides  an  important  motivation  for  the 
undertaking  of  this  project.  Extending  our  previous  study  on  the  subclass  of  linear  programs 
with  linear  complementarity  constraints  (LPCCs),  which  is  essentially  a  linear  program  with 
additional  complementarity  constraints  on  certain  pairs  of  variables,  we  have  investigated 
the  class  of  convex  quadratic  programs  with  complementarity  constraints  (QPCCs)  and  be¬ 
gun  to  explore  the  global  resolution  of  the  class  of  optimization  problems.  Our  work  has 
provided  a  solid  foundation  for  the  design  of  efficient  algorithms  for  obtaining  a  solution  with 
provable  quality  of  the  class  of  highly  challenging  complementarity  constrained  optimization 
problems. 

Publications:  Six  papers  have  been  published;  all  are  available  from  http://www.rpi. 
edu/~mitchj: 

•  L.  Bai,  J.E.  Mitchell,  and  J.S.  Pang.  On  convex  quadratic  programs  with  linear 
complementarity  constraints.  Computational  Optimization  and  Applications  54  (2013)  517- 
544. 

The  paper  shows  that  the  global  resolution  of  a  general  convex  quadratic  program  with  com¬ 
plementarity  constraints  (QPCC),  possibly  infeasible  or  unbounded,  can  be  accomplished 
in  hnite  time.  The  method  constructs  a  minmax  mixed  integer  formulation  by  introduc¬ 
ing  hnitely  many  binary  variables,  one  for  each  complementarity  constraint.  Based  on  the 
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primal-dual  relationship  of  a  pair  of  convex  quadratic  programs  and  on  a  logical  Benders 
scheme,  an  extreme  ray/point  generation  procedure  is  developed,  which  relies  on  valid  satish- 
ability  constraints  for  the  integer  program.  To  improve  this  scheme,  we  propose  a  two-stage 
approach  wherein  the  hrst  stage  solves  the  mixed  integer  quadratic  program  with  pre-set 
upper  bounds  on  the  complementarity  variables,  and  the  second  stage  solves  the  program 
outside  this  bounded  region  by  the  Benders  scheme.  We  report  computational  results  with 
our  method.  We  also  investigate  the  addition  of  a  penalty  term  Dw  to  the  objective 
function,  where  y  and  w  are  the  complementary  variables  and  D  is  a  nonnegative  diagonal 
matrix.  The  matrix  D  can  be  chosen  effectively  by  solving  a  semidefinite  program,  ensuring 
that  the  objective  function  remains  convex.  The  addition  of  the  penalty  term  can  often 
reduce  the  overall  runtime  by  at  least  50%.  We  report  preliminary  computational  testing 
on  a  QP  relaxation  method  which  can  be  used  to  obtain  better  lower  bounds  from  infeasi¬ 
ble  points;  this  method  could  be  incorporated  into  a  branching  scheme.  By  combining  the 
penalty  method  and  the  QP  relaxation  method,  more  than  90%  of  the  gap  can  be  closed  for 
some  QPCC  problems. 

•  L.  Bai,  J.E.  Mitchell,  and  J.S.  Pang.  Using  quadratic  convex  reformulation  to 
tighten  the  convex  relaxation  of  a  quadratic  program  with  complementarity  constraints. 
Optimization  Letters  8  (2014)  811-822. 

Abstract:  Quadratic  Convex  Reformulation  (QCR)  is  a  technique  that  has  been  proposed  for 
binary  and  mixed  integer  quadratic  programs.  In  this  paper,  we  extend  the  QCR  method 
to  convex  quadratic  programs  with  linear  complementarity  constraints  (QPCCs).  Due  to 
the  complementarity  relationship  between  the  nonnegative  variables  y  and  tc,  a  term  y"^Dw 
can  be  added  to  the  QPCC  objective  function,  where  D  is  a  nonnegative  diagonal  matrix 
chosen  to  maintain  the  convexity  of  the  objective  function  and  the  global  resolution  of  the 
QPCC.  Following  the  QCR  method,  the  products  of  linear  equality  constraints  can  also  be 
used  to  perturb  the  QPCC  objective  function,  with  the  goal  that  the  new  QP  relaxation 
provides  a  tighter  lower  bound.  By  solving  a  semidehnite  program,  an  equivalent  QPCC  can 
be  obtained  whose  QP  relaxation  is  as  tight  as  possible.  In  addition,  we  extend  the  QCR 
to  a  general  quadratically  constrained  quadratic  program  (QCQP),  of  which  the  QPCC  is  a 
special  example.  Computational  tests  on  QPCCs  are  presented. 

•  J.  Hlf,  J.E.  Mitchell,  and  J.S.  Pang.  An  LPCC  approach  to  nonconvex  quadratic 
programs.  Mathematical  Programming,  Series  A  133  (2012)  243-277. 

Abstract:  Filling  a  gap  in  nonconvex  quadratic  programming,  this  paper  shows  that  the 
global  resolution  of  a  feasible  quadratic  program  (QP),  which  is  not  known  a  priori  to  be 
bounded  or  unbounded  below,  can  be  accomplished  in  hnite  time  by  solving  two  linear 
programs  with  linear  complementarity  constraints,  i.e.,  LPCCs.  Specihcally,  this  task  can 
be  divided  into  two  LPCCs:  the  hrst  conhrms  whether  the  QP  is  bonnded  below  on  the 
feasible  set  and,  if  not,  compntes  a  feasible  ray  on  which  the  QP  is  nnbonnded;  the  second 
LPCC  compntes  a  globally  optimal  solntion  if  it  exists,  by  identifying  a  stationary  point  that 
yields  the  best  qnadratic  objective  valne.  In  tnrn,  the  global  resolntion  of  these  LPCCs  can 
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be  accomplished  by  a  parameter-free,  mixed  integer-programming  based,  finitely  terminating 
algorithm  developed  recently  by  the  authors,  which  can  be  enhanced  in  this  context  by  a 
new  kind  of  valid  cut  derived  from  the  second-order  conditions  of  the  QP  and  by  exploiting 
the  special  structure  of  the  LPCCs.  Throughout,  our  treatment  makes  no  boundedness 
assumption  of  the  QP;  this  is  a  signihcant  departure  from  much  of  the  existing  literature 
which  consistently  employs  the  boundedness  of  the  feasible  set  as  a  blanket  assumption.  The 
general  theory  is  illustrated  by  3  classes  of  indehnite  problems:  QPs  with  simple  upper  and 
lower  bounds  (existence  of  optimal  solutions  is  guaranteed);  same  QPs  with  an  additional 
inequality  constraint  (extending  the  case  of  simple  bound  constraints);  and  nonnegatively 
constrained  copositive  QPs  (no  guarantee  of  the  existence  of  an  optimal  solution).  We  also 
present  numerical  results  to  support  the  special  cuts  obtained  due  to  the  QP  connection. 

•  J.  Hu,  J.E.  Mitchell,  J.S.  Pang,  and  B.  Yu.  On  linear  programs  with  linear 
complementarity  constraints.  Journal  of  Global  Optimization  53  (2012)  29-51. 

Abstract:  The  paper  is  a  manifestation  of  the  fundamental  importance  of  the  linear  pro¬ 
gram  with  linear  complementarity  constraints  (LPCC)  in  disjunctive  and  hierarchical  pro¬ 
gramming  as  well  as  in  some  novel  paradigms  of  mathematical  programming.  In  addition  to 
providing  a  unihed  framework  for  bilevel  and  inverse  linear  optimization,  nonconvex  piecewise 
linear  programming,  indehnite  quadratic  programs,  quantile  minimization,  and  zero-norm 
minimization,  the  LPCC  provides  a  gateway  to  a  mathematical  program  with  equilibrium 
constraints,  which  itself  is  an  important  class  of  constrained  optimization  problems  that  has 
broad  applications.  We  describe  several  approaches  for  the  global  resolution  of  the  LPCC, 
including  a  logical  Benders  approach  that  can  be  applied  to  problems  that  may  be  infeasible 
or  unbounded. 

•  J.E.  Mitchell,  J.S.  Pang,  and  B.  Yu.  Convex  quadratic  relaxations  of  noncon¬ 
vex  quadratically  constrained  quadratic  programs.  Optimization  Methods  and  Software  29 
(2014)  120-136. 

Abstract:  Nonconvex  quadratic  constraints  can  be  linearized  to  obtain  relaxations  in  a  well- 
understood  manner.  We  propose  to  tighten  the  relaxation  by  using  second  order  cone  con¬ 
straints,  resulting  in  a  convex  quadratic  relaxation.  Our  quadratic  approximation  to  the 
bilinear  term  is  compared  to  the  linear  McCormick  bounds.  The  second  order  cone  con¬ 
straints  are  based  on  linear  combinations  of  pairs  of  variables.  With  good  bounds  on  these 
linear  combinations,  the  resulting  constraints  strengthen  the  McCormick  bounds.  Computa¬ 
tional  results  are  given,  which  indicate  that  the  convex  quadratic  relaxation  can  dramatically 
improve  the  solution  times  for  some  problems. 

•  J.E.  Mitchell,  J.S.  Pang,  and  B.  Yu.  Obtaining  tighter  relaxations  of  mathematical 
programs  with  complementarity  constraints.  In  T.  Terlaky  and  F.  Curtis,  editors.  Modeling 
and  Optimization:  Theory  and  Applications.  Springer  Proceedings  in  Mathematics  and 
Statistics  (New  York  2012)  Volume  21,  pp.  1-23. 

Abstract:  The  class  of  mathematical  programs  with  complementarity  constraints  (MPCCs) 
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constitutes  a  powerful  modeling  paradigm.  In  an  effort  to  find  a  global  optimum,  it  is  often 
useful  to  examine  the  relaxation  obtained  by  omitting  the  complementarity  constraints.  We 
discuss  various  methods  to  tighten  the  relaxation  by  exploiting  complementarity,  with  the 
aim  of  constructing  better  approximations  to  the  convex  hull  of  the  set  of  feasible  solutions  to 
the  MPCC,  and  hence  better  lower  bounds  on  the  optimal  value  of  the  MPCC.  Better  lower 
bounds  can  be  useful  in  branching  schemes  to  hnd  a  globally  optimal  solution.  Different 
types  of  linear  constraints  are  constructed,  including  cuts  based  on  bounds  on  the  variables 
and  various  types  of  disjunctive  cuts.  Novel  convex  quadratic  constraints  are  introduced, 
with  a  derivation  that  is  particularly  useful  when  the  number  of  design  variables  is  not  too 
large.  A  lifting  process  is  specialized  to  MPCCs.  Semidehnite  programming  constraints  are 
also  discussed.  All  these  constraints  are  typically  applicable  to  any  convex  program  with 
complementarity  constraints.  Computational  results  for  linear  programs  with  complemen¬ 
tarity  constraints  (LPCCs)  are  included,  comparing  the  beneht  of  the  various  constraints  on 
the  value  of  the  relaxation,  and  showing  that  the  constraints  can  dramatically  speed  up  the 
solution  of  the  LPCC. 

Two  additional  papers  have  been  submitted  for  publication: 

•  L.  Bai,  J.E.  Mitchell,  and  J.S.  Pang.  On  QPCCs,  QCQPs  and  completely  positive 
programs.  Mathematical  Programming,  Series  A,  submitted  January  2014. 

In  this  paper,  we  show  that  any  quadratically  constrained  quadratic  program  is  equivalent 
to  a  convex  optimization  problem.  The  result  requires  no  assumptions  on  the  boundedness 
of  the  feasible  region  or  on  the  convexity  of  the  quadratic  constraints.  The  result  also  holds 
if  a  nonnegativity  assumption  on  the  variables  is  replaced  by  a  more  general  convex  conic 
constraint;  for  example,  the  result  holds  if  the  variables  satisfy  a  second  order  cone  constraint 
or  if  the  variables  satisfy  a  semidehniteness  constraint.  The  proof  exploits  the  relationship 
between  this  class  of  problems  and  quadratic  programs  with  complementarity  constraints. 

The  results  show  that  many  important  practical  problems  can  be  represented  as  convex 
optimization  problems  through  the  use  of  a  single  lifting.  This  includes  rank-constrained 
semidehnite  programs.  The  latter  class  includes  problems  such  as  factor  analysis  problems 
for  hnding  a  low-rank  covariance  matrix,  sensor  array  processing,  minimizing  the  rank  of 
a  Hankel  matrix  in  model  identihcation  problems  in  system  theory  and  signal  processing, 
problems  in  systems  and  control,  and  combinatorial  optimization  problems.  The  convex 
optimization  problem  is  not  easy  to  solve,  because  it  is  dehned  over  a  convex  cone  that  is 
hard  to  work  with.  Nonetheless,  the  exact  reformulation  offers  the  possibility  of  designing 
new  and  effective  algorithms  for  solving  a  broad  class  of  optimization  problems. 

•  M.  Feng,  J.E.  Mitghell,  J.S.  Pang,  X.  Seen,  and  A.  Waeghter.  Complemen¬ 
tarity  formulations  of  ^Q-norm.  optimization  problems.  Mathematical  Programming,  Series 
A,  submitted  September  2013. 

In  this  paper,  we  developed  a  nonlinear  programming  formulation  of  the  problem  of  min¬ 
imizing  the  number  of  nonzero  components  in  the  solution  of  a  system  of  linear  equalities 
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and  inequalities,  also  known  as  minimizing  the  iQ-noim..  This  is  the  problem  of  compressed 
sensing  that  is  currently  of  great  interest.  It  also  arises  in  the  search  for  sparse  solutions 
to  support  vector  machine  problems  in  classihcation.  Our  method  is  slower  than  a  linear 
programming  approximation  to  the  io-norm  problem,  but  it  has  the  advantage  that  it  often 
gives  a  sparser  solution  than  the  L  1-norm  approximation.  This  seems  to  be  especially  true 
when  the  system  involves  inequality  constraints,  as  is  the  case  for  sparse  support  vector 
machines. 

The  following  doctoral  thesis  supervised  by  the  PI  was  successfully  defended  in  December 
2012: 

•  Yuching  Lee.  Global  Solution  to  Parametric  Complementarity  Constrained  Programs 
and  Applications  in  Optimal  Parameter  Selection.  Ph.D.  thesis.  Department  of  Industrial 
and  Enterprise  Systems  Engineering,  University  of  Illinois  at  Urbana-Champaign  (December 
2012). 

Further  papers  are  in  preparation,  including  documentation  of  methods  for  determining 
global  optima  for  cross-validation  problems,  and  improved  methods  for  Ending  global  optima 
of  problems  with  complementarity  constraints.  Multiple  talks  have  been  given  at  conferences 
and  universities  by  students  and  collaborator  Mitchell. 
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